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We study the effect of varying wiring in excitable random networks in which connec- 
tion weights change with activity to mold local resistance or facilitation due to fatigue. 
Dynamic attractors, corresponding to patterns of activity, are then easily destabilized 
according to three main modes, including one in which the activity shows chaotic hop- 
ping among the patterns. We describe phase transitions to this regime, and show a 
monotonous dependence of critical parameters on the heterogeneity of the wiring distri- 
bution. Such correlation between topology and functionality implies, in particular, that 
tasks which require unstable behavior — such as pattern recognition, family discrimination 
and categorization — can be most efficiently performed on highly heterogeneous networks. 
It also follows a possible explanation for the abundance in nature of scale-free network 
topologies. 



PACS numbers: 64.60. an, 05.45.-a, 84.35. -l-i, 87.19.lj 

Excitable systems allow for the regeneration of waves propagating through them, and may thus 
respond vigorously to weak stimulus. The brain and other parts of the nervous system are well- 
studied paradigms, and forest fires with constant ignition of trees and autocatalytic reactions in 
surfaces, for instance, also share some of the basics (Q; S; S; 0; S)- The fact that signals are not 
gradually damped by friction in these cases is a consequence of cooperativeness between many 
elements in a nonequilibrium setting. In fact, the systems of interest may be viewed as large 
networks whose nodes are "excitable". This, which admits various realizations, typically means 
that each element has a threshold and a refractory time between consecutive responses. 

Brain tasks may ideally be reproduced in mathematical neural networks. These consist of 
neurons — often simplified as binary variables — connected by edges representing synapses (0; 
0; S)- Assuming edges weighted according to a prescription (e.g. (@)) which in a sense saves 
information from a set of given patterns of activity, these patterns become attractors of the phase- 
space dynamics. Therefore, the system shows retrieval of the stored patterns, known as associative 
memory. Actual neural systems do much more than just recalling a memory and staying there, 
however. That is, one should expect dynamic instabilities or other destabilizing mechanism. This 
expectation is reinforced by recent experiments suggesting that synapses undergo rapid changes 



with time which may both determine brain tasks ([lO|; [ill; [l^; Il3l ) and induce irregular and perhaps 
chaotic activity ( 14: fisl). 



The observed rapid changes (which have been described (jlll : Il2l : Il3l ) as causing "synaptic 
depression" and/or "facilitation" on the time scale of milliseconds — i.e., much faster than the 
plasticity process in which synapses store patterns) may correspond to the characteristic behavior 
of single excitable elements. Furthermore, a fully-connected network which describes cooperation 
between such excitable elements exhibits both attractors and chaotic instabilities (fl^ ). Here, we 
extend and generalize this study to conclude on the influence of the excitable network topology 
on dynamic behavior. We show, in particular, an interesting correlation between certain wiring 
topology and optimal functionality. 

Consider N binary nodes, Sj = ±1, and the topology matrix, €ij = 1,0, which indicates the 
existence or not of an edge between nodes i,j = 1,2,...,N. Let a set of M patterns, = ±1, 
u = 1, ...M (which we generate here at random), and assume that they are "stored" by giving 
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each edge a base weight UiJ = . Actual weights are dynamic, however, e.g., coij = 

TJijXj where Xj is a stochastic variable. Assuming the limit in which this varies in a time scale 
infinitely smaller than the one for node dynamics, we may consider a stationary distribution such 
as P{xj\S) = q6{xj — Ej) + (1 — q)5{xj — 1), 5 = {sj} , for instance. This amounts to assume that, 
at each time step, every connection has a probability q of altering its weight by a factor Hj which 
is a function (to be determined) of the local field a t 7, namely, the net current from other nodes. 
This choice differs essentially from the one in Ref. ijla ). where q depends on the global degree of 
order and is a constant independent of j. 

Assume independence of the noise at different edges, and that the transition rate for the stochas- 
tic changes is 

c (5 ^ y ) _ -p|- J dxjP{xj\S)^{uij) 
HS^^S) - ii Jdx,P{x,m^{-u,,y 

where Uij = SiSjXjZJijT^^ , ^(tt) = exp(— ^ti) to have proper contour conditions, T is a "tem- 
perature" parameter, and stands for S after the change Si — > — Sj. (For a description of this 
formalism and its interpretation, see 113) ■) We define the effective local fields = h^^(S, T, q) via 
l\j ^ij/'ftj = exp {-hfsi/T) , where (pf^ = qexp {±EjVij) + {I - q) exp {±Vij), with Vij = \eijUij. 
Effective weights oj^J then follow from hf^ = "^jOJ^Jsjeij. To obtain an analytical expression, we 
linearize around TJij = (a good approximation when M <C A^), which yields 

a;|;=[l + g(H,-l)]a^. 

In order to fix here, we first introduce the overlap vector ni = {m} , ...m^^), with my = 
A^^^^^j^Sj, which measures the correlation between the current configuration and each of the 
stored patterns, and the local one m| of components = {k)~'^ Yli Cr^i^jh where (k) is the mean 
node connectivity, i.e., the average of /cj = Ylj ^ij- We then assume, for any q ^ 0, that the factor 
is Ej = l + C(/ip(^ - l)/q, with 

c(/ij) = x7(i+M/A) j^i/ijr 

V 

where x = A/ (A;) and a > is a parameter. This comes from the fact that the field at node j may 
be written as a sum of components from each pattern, namely, hj = X]^^ where 

i 

Our choice for Hj, which amounts to assume that the "fatigue" at a given edge increases with the 
field at the preceding node j (and allows to recover the fully-connected limit in (0) if a = 2), 
finally leads to 

u;^f =[l + (c^-l)0(ml)]aJ^. 

Varying <I> one sets the nature of the weights. That is, < $ < 1 corresponds to resistance 
{depression) due to heavy local work, while the edge facilitates, i.e., tends to increase the effect of 
the signal under the same situation for $ > 1. (The action of the edge is reversed for negative $.) 
We performed Monte Carlo simulations using standard parallel updating with the effective rates 
c (5* — > 5*) computed using the latter effective weights. 

It is possible to solve the single pattern case (M = 1) under a mean-field assumption, which 
is a good approximation for large enough connectivity. That is, we may substitute the matrix e^j 
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by its mean value over network realizations to obtain analytical results that are independent of 
the underlying disorder. Imagine that each node hosts ki half-edges according to a distribution 
p{k), the total number of half-edges in the network being {k)N. Choose a node i at random and 
randomly join one of its half-edges to an available free half-edge. The probability that this half- 
edge ends at node j is kj/ ((A;) A^) . Once all the nodes have been linked up, the expected value 
(as a quenched average over network realizations) for the number of edges joining nodes i and j 
is eJJ = kikj/ {{k) N) (0). Using the notation r]i = ^iSi, we have wu = x{'ni^ij)i = ^Y^iVi^ij- 
Because node activity is not statistically independent of connectivity (|2ll ). we must define a new set 
of overlap parameters, analogous to m and rrij. That is, = {k^r]i)i/ [k^) and the local versions 
^■'n = x{kf'ni^ij)i/{f^^)- After using eij = lij, one obtains the relation = {k^^^)kiiJ,n+i/{{k'^){k)'^). 
Inserting this expression into the definition of and substituting (sj) = tanh[T~^h^^^ (S)] (for 
very large N), standard mean- field analysis yields 

I'nit + l) = -^(A;"tanhMT,#(A;,t))^, 



where the last quantity is defined as 

k 



Mt<s> 



TN 



/.i(t) + (c^-i)^^i/ii(t)r/i,+i(t) 



{k) 



This is a two-dimensional map which is valid for any random topology of distribution p{k). Note 
that the macroscopic magnitude of interest is /xq = m = |r??|. 

A main consequence of this is the existence of a critical temperature, Tc, under very general 
conditions, e.g., for many different network connectivities. More specifically, as T is decreased, 
the overlap m describes a second-order phase transition from a disordered or, say, "paramagnetic" 
phase to an ordered ("ferromagnetic") phase which exhibits associative memory. The mean-field 
temperature signaling this transition is 

T,= {k'){{k)N)-\ 

On the other hand, the map reduces for T = to /i„ (t + 1) = 
sign [^n (t) [l + (^ - l)(fc)°+^/(/c°+^)] } . This implies the existence at <i> = <I>o, where 

$0 = 1- (A;)"+V(A;°+i), 

of a transition as $ is decreased from the ferromagnetic phase to a new phase in which periodic 
hopping between the attractor and its negative occurs. This is confirmed by the Monte Carlo 
simulations for M > 1, namely, the hopping is also among different attractors for finite T. The 
simulations also indicate that this transition washes out at low enough finite temperature. Instead, 
actual evolutions show that, for a certain range of $ values, the system activity then exhibits 
chaotic behavior. 

The transition from ferromagnetic to chaotic states is a main concern hereafter. Our interest 
in this regime follows from several recent observations concerning the relevance of chaotic activity 
in a network. In particular, it has been shown that chaos might be responsible for certain states 



of attention duringbrain activity (|2^; |23| ) , and that some network properties such as the compu- 
tational capacity (2J) and the dynamic range of sensitivity to stimuli (j25l ) may become optimal at 
the "edge of chaos" in a variety of settings. 

We next notice that the critical values Tc and $0 only depend on the moments of the generic 
distribution p{k), and that the ratio {k'^) / {k)"" , a > 1, is a convenient way of characterizing het- 
erogeneity. We studied in detail two particular types of connectivity distributions with easily 
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tunable heterogeneity, namely, networks with {k)N/2 edges randomly distributed with p (k) such 
that the heterogeneity depends on a single parameter. Our first case is the bimodal distribution, 
p{k) = ^S{k — ki) + ^S{k — /C2) with parameter A = {k2 — ki)/2 = (k) — ki = k2 — {k). Our second 
case is the scale-free distribution, p{k) ~ k~'^, which does not have any characteristic size but k 

is confined to the limits, ko and km < min(A;oA^^-i , — 1) for finite N (0). Notice that the net- 
work in this case gets more homogeneous as 7 is increased (j27l ) , and that this kind of distribution 



seems to be most relevant in nature (j20l : l21l : l28l : |29| ). In particular, the functional topology in the 
human brain, as defined by correlated activity between small clusters of neurons, has been shown 
to correspond to this case with exponent 7 ~ 2 (jsol ) (in spite of the fact that the brain's structural 
or wiring topology is not yet well known). 

We obtained the critical value of the fatigue, $c (T) , from Monte Carlo simulations at finite 
temperature T. These indicate that chaos never occurs for T > 0.35Tc. On the other hand, a 
detailed comparison of the value <I>c with $0 — as obtained analytically for T = — indicates that 
$c — ^0- Figure [1] illustrates the "error" $0 ~ (T) for different topologies. This shows that the 
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FIG. 1 The temperature dependence of the difference between the values for the fatigue at which the 
ferromagnetic-periodic transition occurs, as obtained analytically for T = (<i>o) and from MC simulations 
at finite T (<i>c)- The critical temperature is calculated as — (fc^) ((fc) iV) for each topology. Data are 
for bimodal distributions with varying A and for scale-free topologies with varying 7, as indicated. Here, 
(fc) = 20, N = 1600 and a = 2. The bars are standard deviations for 10 network realisations. 

approximation ^>c — ^"0 is quite good at low T for any of the cases examined. Therefore, assuming 
the critical values for the main parameters, Tc and $0, as given by our map, we conclude that 
the more heterogeneous the distribution of connectivities of a network is, the lower the amount of 
fatigue, and the higher the critical temperature, needed to destabilize the dynamics. As an example 
of this interesting behavior, consider a network with {k) = In(A^), and dynamics according to a = 2. 
If the distribution were regular, the critical values would be Tc = ln{N)/N (which goes to zero in 
the thermodynamic limit) and $0 = 0. However, a scale-free topology with the same number of 
edges and 7 = 2 would yield Tc = I and $0 = 1 — 2{ln N)^/N'^ (which goes to 1 as iV — > 00). 

Figure [2] illustrates, for two topologies, the phase diagram of the ferromagnetic-chaotic transi- 
tion. Most remarkable is the plateau observed in the edge-of-chaos or transition curve for scale-free 
topologies around 7 ~ 2, for which very little fatigue, namely, 'I' ^ 1 which corresponds to slight 
depression, is required to achieve chaos. The limit 7 ^ cxo corresponds to (A:)-regular graphs 
(equivalent to A = 0). If 7 is reduced, km increases and /cq decreases. The network is truncated 
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FIG. 2 The critical fatigue values $0 (solid lines) and <i>c from MC averages over 10 networks (symbols) 
with T = (fc) = 20, N — 1600, a — 2. The dots below the lines correspond to changes of sign of the 

Lyapunov exponent as given by the iterated map, which roughly agree with the other results. This is for 
bimodal and scale-free topologies, as indicated. 

when km = N . It follows that a value of 7 exits at which cannot be smaller, so that km must 
drop to preserve {k). This explains the fall in $c as 7 ^ 1. 

As a further illustration of our findings, we monitored the performance as a function of topology 
during a simulation of pattern recognition. That is, we "showed" the system a pattern, say v chosen 
at random from the set of M previously stored, every certain number of time steps. This was 
performed in practice by changing the field at each node for one time step, namely, hi ^ hi + 5^^, 
where 5 measures the intensity of the input signal. Ideally, the network should remain in this 
configuration until it is newly stimulated. The performance may thus be estimated from a temporal 
average of the overlap between the current state and the input pattern, {m'^)time- This is observed 
to simply increase monotonically with A for the bimodal case. The scale-free case, however, as 
illustrated in figure [3l shows how the task is better performed the closer to the edge of chaos the 
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FIG. 3 Network "performance" (see the main text) against 7 for scale-free topology with $ = 1, as an 
average over 20 network realizations with stimulation every 50 MC steps for 2000 MC steps, S ~ 5 and 
M = A; other parameters as in Fig. [2l Inset shows sections of typical time series of m'^ for 7 = 4; the 
corresponding stimulus for pattern 1/ is shown below. 

network is. This is because the system is then easily destabilized by the stimulus while being 
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able to retrieve a pattern with accuracy. Figure [3] also shows that the best performance for the 
scale-free topology when $ = 1, i.e., in the absence of any fatigue, definetely occurs around 7 = 2. 

The fact that the model network above is one of the simplest situations one may conceive — 
with dynamic connections which depend on local fields — suggest that topological heterogeneity 
may indeed be a relevant property for a complex network to perform efficiently certain high level 
functions. This has in practice been illustrated before using networks with a similar philosophy 
which happen to be useful for pattern recognition and class identification (l3ll ). Our system retrieves 
memory patterns with accuracy in spite of noise, and yet it may easily destabilize itself to change 
state in response to an input signal — without requiring an excessive fatigue for the purpose. 
There is a correlation between the amount $ of fatigue and the value of 7 for which performance is 
maximized. One may argue that the plateau of "good" behavior shown around 7 ~ 2 for scale-free 
networks with <I> < 1 (figure [2|) is a possible justification for the supposed tendency of certain 
systems in nature to evolve towards this topology. It may also serve as a hint when implementing 
artificial networks. 

We thank M.A. Muhoz for very helpful comments. This work was financed by the JA project 
FQM-01505 and by the MEC-FEDER project FIS2005-00791. 
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